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ABSZBACT 

This publication presents a brief overview and 
discussion of the Educational Leadership Appraisal (ELA) system, 
which was developed for use in Nova University's National Ed«D« 
Program by Education Besearch Corporation* ELA is a p.erf ormance-based 
procedure for observing and assessing leadership behavior that 
evaluates an individual's potential for exercising leadership in an 
administrative position within a school system* Each individual is 
appraised along 23 leadership dimjensiqn£^# which are grouped into the 
broad categories of management and organization, communication, 
problem-*solving, task orientation, and interpersonal qualities* 
(JG) 



* Documents acquired by EfilC include many informal unpublished * 
^ materials not available from otlier sources* EBIC makes every effort * 

* to obtain the best copy available* Nevertheless, items of marginal * 

* reproducibility are often encountered and this affects the quality * 

* of the microfiche and hardcopy reproductions EBIC makes available * 
^ via the EBIC Document Be production Service (EDBS) • EDBS is not * 
^ responsible for the quality of the original document* Beproductions * 

* supplied by EDBS are the best that can be made from the original*. ^ 
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Those who took part in 
arranging for Allan Ellis to 
discuss the Educational 
Leadership Appraisal 
(ELA ) concept at Sum- 
mer Institute 74 were grati- 
fied, but not at all surprised, 
at the Intensity of interest 
his presentation evoked. 
Participants who attended 
Dr. Ellis' workshop-discus- 
sions were prompt to con- 
cede that their barrages of 
questions drew his meet- 
ings well beyond their 
alloted time and that it was 
with some reluctance that 
participants allowed him to break 
away to meet other groups. 

This interest in ELA, on both 
professional and personal levels, 
was expected. Institutions and their 
staffs, in education and elsewhere, 
have for years pondered and at- 
tempted to answer the questions: 
what are the qualities that make a 
leader? how do you identify and 
assess these qualities? how can 
you develop them? In the business 
world the question has been: what 
kinds of people are successful in 
executive positions? In education, 
where "success" is more difficult 
to define, school systems have 
nevertheless continued to search 
for a systematic means of identify- 
ing the components and patterns 
of administrative leadership. 
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implications for the better- 
ment of education. We be- 
lieve, too, that ELA can 
provide data that will enable 
us to optimize the National 
Ed.D. Program, and to 
identify components and 
content of the program 
that will best foster the 
development of leadership 
in school administration. 

For these reasons— and 
especially because the Na- 
tional Ed.D. Program ad- 
dresses itself to the interests 
and concerns of those 
school administrators who 
5eek and exercise leadership in 
he public school systems— ELA 
vill be used with all entering Group 
I participants as a normal com- 
ponent of the Ed.D. Program. The 
aim of this report, therefore, is to 
nform readers of the Gazette about 
his important venture: to explain 
vhat ELA is, how it works, and 
vhat it is expected to mean for 
he program, its participants, and 
he educational community. 

<A.\-V;]uM U Is 

The Educational Leadership Ap- 
iraisal system was developed for 
Jova University's National Ed. D. 
Yogram by Educational Research 
Corporation, Watertown, Massa- 
husetts, of which Dr. Ellis is 
resident. It is based in part on 





Performance-based, ELA examines and assesses leadership behavior through individual exercises,., group 
exercises . . . simulations , , , role-playing , , . analyses problems , . . writing and speaking assignments. 
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techniques that have been suc- 
cessfully employed in government 
and industry for two decades. Un- 
like the conventional techniques 
of supervisor's ratings, personal 
interviews, and pencil-and-paper 
examinations, ELA is a performance- 
based procedure for the observa- 
tion and assessment of leadership 
behavior. It focuses on potential 
to exercise leadership in an ad- 
ministrative position within a school 
system, appraising the adminis- 
trator along twenty-three Leader- 
ship Dimensions, grouped into the 
broader categories of Manage- 
ment and Organization. Commu- 
nication, Problem Solving. Task 
Orientation, and Interpersonal Qual- 
ities. These twenty-three dimensions 
have been selected as encom- 
passing the aptitudes, abilities, 
competencies, and qualities de- 
manded by the school adminis- 
trator's job. 



Behavior along the Leadership 
Dimensions is elicited through a 
set of individual and group exer- 
cises, simulations, role-playing tasks, 
analysis problems, interviews, and 
writing and speaking assignments 
that approximate the problems, 
issues, and challenges encoun- 
tered by today's school adminis- 
trator. Appraisal begins with the 
systematic collection of the elicited 
behavior, accomplished at a full- 
day session at which participants 
are videotaped as they work their 
way through the various assign- 
ments. To these tapes is added 
written material produced by the 
participant both at home and at a 
previous day-long session. Then 
a team of appraisers at the Boston 
headquarters of Educational Re- 
search Corporation studies the 
tapes and written materials to 
produce a Behavior Inventory — a 
listing, by Leadership Dimensions. 
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Management and 

Organization 

Planning and Organization— Effective- 
ness in planning and organizing own 
activities and those of staff. 

Management Control— The appreci- 
ation of needs for controls and the 
ability to maintain controls over 
processes. 

Use of Delegation— The ability to use 
subordinates, colleagues, and super- 
iors effectively and to understand 
when a decision can best be made. 

Communication 

Written Communication Skills--The 
ability to write a readable, persuasive 
document that conceptualizes Issues 
and problems, is complete, and is 
appropriate for the intended audience. 

Oral Communication Skills— The ability 
to respond to questions in public, 
think clearly on one's feet, think 
behind questions, be articulate in 
defense of programs, define mis- 
sions, explain complex issues, deal 
with a problem in public without prep- 
aration, and relate to all levels 
of audiences. 

Problem Solving 

Problem Analysis— The ability to read 
or listen and pick out critical informa- 
tion or issues underlying the material; 
to seek out pertinent data; to weigh 
alternatives and their consequences. 




Leader.ship 
Dimensions in ELA 





Political Analysis— The ability to per- 
ceive critical features of the environ- 
ment such as power structure, principal 
players, special Interests of individuals 
and groups; to be aware of con- 
straints on ability to function; to 
formulate alternatives for action and 
bases for compromise that reflect 
realistic expectations, I.e., a sense of 
what Is do-able. 

Judgment— The ability to reach log- 
ical conclusions based on the evidence 
at hand. 

Risk Taking— The extent to which 
calculated risks are taken based on 
sound judgment. 

Creativity— The ability to come up 
with imaginative solutions in manage- 
ment situations. 

Knowledge of Education— The extent 
of knowledge and understanding of 
fundam'ental as well as current edu- 
cational principles and issues. 

Task Orientation (Alone) 

Tenacity— The tendency to stay with a 
problem or line of thought until' the 
matter Is settled. 



TV camera watches pariicipant al simu- 
lated news conference, records behavior 
(or analysis by ERC appraisers. 



of all relative behavior exhibited 
by the person being assessed. 
This Inventory provides, in turn, 
the basis for an overall Perfor- 
mance Analysis conducted by an 
augmented team of appraisers. 

From this analysis the appraisal 
team produces concensus ratings 
and descriptions, which the senior 
appraiser then Incorporates Into 
an extensive written statement, or 
Leadership Profile, depleting and 
characterizing the candidate's 
performance on the components 
of the ELA system. 

It should be evident that ELA is 
a comprehensive system that re- 
quired substantial funds for devel- 
opment. All the costs Incurred In 



Initiative— Active efforts to influence 
events rather than passive acceptance. 

Stress Tolerance— Stability of perfor- 
mance under conditions of pressure, 
opposition, and uncertainty. 

Energy— The ability to sustain a high 
level of work activity. 

Task Orientation 
(With Others) 

Group Leadership— Effectiveness in 
bringing a group to accomplish a task. 

Individual Leadership— Ability to moti- 
vate individuals on a one-to-one basis 
to accomplish a task. 

Adaptability— The ability to modify 
behavioral style and management 
approach to reach a goal. 

Decisiveness— Readiness to make deci- 
sions and to render judgment. 

Interpersonal Qualities 
Impact— The ability to create a good 
first impression, to command atten- 
tion and respect, to show an air of 
confidence, and to achieve per- 
sonal recognition. 

Flexibility— Being receptive to new 
ideas or arguments. 

Perceptlveness— Skill in perceiving 
minimal cues in the behavior of others. 

Empathy-The ability to Identify with, 
and consider, the feelings, thought, 
and attitudes of another. 



ELA*s development have been paid 
for out of operating funds by 
Nova's National Ed. D. Program. 
Included in these costs, which total 
over $125,000 to date, have been 
those for a full-scale four-month 
field test of the system, appraising 
approximately 100 participants from 
the Program's Chicagor Richmond,^ 
Virginia; Philadelphia; and Fair- 
field, California clusters — school 
administrators whose positions 
range from assistant principals to 
superintendents of schools. The 
results of these tests have satis- 
fied the staffs of both the Nova 
Ed. D. Program and Educational 
Research Corporation that the 
ELA system affords the requisite 
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degree of value, veracity, and the 
freedom from minority group bias 
to warrant its use as a component 
of the Ed.D. Program. 

Leadership 

Dimensions in ELA 

Those participants who attended 
Dr. Ellis' presentations during the 
Summer Institute will recall the 
heated debate and discussion 
that arose from efforts of the 
group to develop a useable list of 
qualities and characteristics that 
could be termed Leadership Di- 
mensions. Typically, the lists that 
appeared on the blackboard as 
the various suggestions were re- 
corded ranged from "manage- 
ment skills" to "empathy," from 
"communication skills" to "hon- 
esty." In the process of develop- 
ing such a list, participants found 
themselves wrestling with ques- 
tions of overlap, vagueness, re- 
dundancy, and semantics, as well 
as the validity, if any, of the sug- 
gested dimensions. While there 
was debate about the substance 
of the list, there was, neverthe- 
less, agreement that a useful list 
could be developed, given time, 
experience, and an acute percep- 
tion of the special demands of 
the school administrator's job. 

These, in fact, were the ele- 
ments that were drawn upon in 
the process of developing the 
twenty-three Leadership Dimen- 
sions that constitute the first of 
ELA's several system components. 
Creating the list involved a proc- 
^ess of repeated cut-and-try, using 
a multiplicity of recommendations 
from school administrators, inter- 
spersed with critical scrutiny, sub- 
stitutions, re-examination, and 
re-listing. In addition to the work 
by the Nova and ERC staffs, the 
process entailed the collaboration 
of a number of experts, including, 
Gregory Anrig, Commissioner of 
Education for the Commonwealth 
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The candidate's behavior along the Leadership Dimensions In the 
accompanying list on page 3 is elicited through the series of individual and 
group exercises, simulations, role-playing tasks, problem analyses, and 
interviews described here in abridged form. 

Administrative Action Exercise— Many critical problems and 
challenges of an upper-middle-level administrator are simulated in 
this exercise wherein the participant is asked to deal with the 
accumulated notes, memoranda, letters, requests, and other 
pending action items that confront him as he takes over a new 
position as Area Administrator. The issues represented by the 30 
items in this exercise cover the range from day-to-day internal 
management problems to questions of policy and planning. In 
dealing with these issues the participant must make decisions, 
delegate responsibility, write letters, assign work, plan, organize, 
and schedule. 

Creativity Exercise— The participant is presented with a proposal 
that would affect every classroom in the school district. He or she 
isMhen asked to provide creative solutions/actions regarding 
the proposal. 

Analysis Problem/Written Presentation-The participant is given 
a rather complex proposal regarding a pilot program for a school 
district. The participant must explain this proposal by four separate 
letters to the students, faculty, administration, and Advisory Board. 

(During the two-week period between Days 1 and 2 of ELA, the 
participant creates an extensive experimental project to explore 
new methods of school administration. This exercise in admin- 
istrative innovation forms the basis of the separate but related 
exercises presented in the following four descriptions.) 
individual Oral Presentation— Based on an outline of the pro- 
posed project, the participant delivers a prepared speech designed to 
present, concisely and persuasively, the project's critical fea- 
tures and establish its priority and relevance. 
Question, and Answer Period— Simulating the atmosphere and 
pressures of an unscheduled press conference, this^exercis.e 
calls on the participant to answer questions and respon'd to chal- 
lenges **on his feet" concerning his project and its implications. 
Leaderiess Group Discussion^Unasslgned Roles- Six partici- 
pants, ail of whom have undergone the previous exercises, meet 



Role playing: simulated city council 
meeting stages inter-department bar- 
gaining for available funds. 
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to determine which proposal, If any, Is clearly superior and 
worthy :f funding or, failing that, whether they can collectively 
combine features of the various proposals to create a new, 
eminently fundable and worthwhile project. 
. Writing Exerclse-The participant writes a full proposal for his 
project including rationale, description, management plan, 
budget, required resources, objectives, evaluation procedures, 
relevant contingencies, and the like. 
Analysis Problem/Oral Presentation— The participant is given 
all of the data of a school program that had been collected by his 
predecessor. 7he participant Is to analyze the data, provide 
an outline of his/her evaluation of the program and recommenda- 
tions for Its future operation. Upon submitting this outline he/she 
will make a ten minute oral presentation of the analysis to the 
school board and answer any of their questions. 

Individual Fact Finding/Decision l\ffaki»g Problem-Participants 
are presented with a problem situation and asked to determine its 
cause or causes. They may ask questions of a resource person 
who answers only those questions asked. At the end of the day 
the participants are aisked to develop their conclusions based on 
the information acquired. 

Leaderless Group Discussion— Assigned Roles-The city council 
is meeting to allocate a one-million dollar federal grant for capital 
improvement. Each member of the council (participants) has been 
briefed by a major department head, e.g., streets, fire, sanitation, 
and is instructed to try to get as much money as possible for the 
department. The council must allocate the money In the time 
allowed. The exercise provides participants with a choice of 
projects to back and the opportunity to bargain and trade off 
projects for support. 

Observation /Judgment Exercise— For 20 minutes the participant 
views a video tape of a committee meeting. The purpose of the 
committee is to make recommendations to the school board on 
a controversial subject. The participant acts as a newly-appointed 
Area Administrator who desires a favorable recommendation from 
the committee. After viewing the tape the participant is asked 
questions regarding his future actions toward members of the 
committee and his future role with the committee. 



Of Massachusetts; [VIedill Bair, Ex- 
ecutive Director, EDCO, Inc.; and 
fVlark Shedd, Commissioner of 
Education for the State of Con- 
necticut, working together and 
separately over many weeks. From 
this cooperative effort and cross- 
criticism came a list that Nova, 
Educational Research Corporation, 
and the consulting team could 
agree upon as encompassing the 
spectrum of qualities and attri- 
butes required of educational lead- 



ers in a modern school setting. 
These Leadership Dimensions are 
listed and defined in the panel on 
page 3. 

Although the developers of these 
Leadership Dimensions are satis- 
fied that the list represents our 
best present understanding of 
the skills and aptitudes associ- 
ated with leadership, they recog- 
nize that future experience with 
ELA may indicate the need for 
expanding that list. The structure 
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of the ELA system has therefore 
been kept open to the insertion 
of additional dimensions, such as 
"moral reasoning," ''need for 
achievement,*' "range of interests," 
and "carreer expectations," should 
the need arise. 




Alone with the video recorder, partici- 
pant delivers a ten-minute speech to 
demonstrate oral communication skills, 
audience impact. 



Exercises and 
Behavior Simulations 

One of the several characteriz- 
ing features of the ELA system is 
the use of situational techniques 
to approximate the problems and 
challenges faced by the adminis- 
trator in his job setting. ELA con- 
centrates on the assessment of 
on-the-job behavior rather than 
the application of some theoretical 
set of criteria. 

For the development of exer- 
cises that would provide authentic 
simulation of job problems, ap- 
propriately refined and organized 
to elicit leadership behavior. Edu- 
cational Research Corporation 
drew upon the expertise of De- 
velopmental Dimensions, Inc., 
Pittsburgh, Pennsylvania, an in- 
ternational pioneer in the use of 
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Persuasion: wall chart is used by par- 
ticipant to help make his point. 

this technique in government, and 
industry. The various exercises 
and simulations employed in the 
ELA system are described in 
brief on pages 4 and 5. The ac- 
companying Dimension /Exercise 
Matrix shows the dimensions 
measured by each of the exer- 
cises. As the fTiatrix indicates, 
each of the Leadership Dimen- 
sions is measured via several ex- 
ercises or simulations, providing a 
broader perspective than couic! 
be obtained from a one-to-one 
correspondence of exercise and 
dimension, and eliminating the 
danger of yes/no appraisals that 
could result from an attempt to 
assess a behavior from a single 
exercise. 

Administration of these exer- 
cises in ELA's present mode is a 
two-day process. Participants first 
go through one day of written exer- 
cises: the administrative action 
exercise, creativity problem, and 
the writing exercise. During that 
day they are given two tal<e-home 
assignments: preparation of the 
oral presentation and completion 
of the Background Information 
Form. Two weeks later they go 
through a second day of exer- 
cises: the two leaderless group 
discussions, two speeches, the 
fact-finding exercise, the staffing 
analysis problem, the simulated 
press conference, and the analysis 
of a community task force meeting. 
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Although the administration of 
the ELA exercises is described as 
a two-day process, in reality these 
exercises are of a range and sub- 
stance that would normally be 
expected to require three working 
days for completion. They have 
been intentionally crammed into 
two days in order to create a non- 
leisure situation that compels the 
participant to keep several prob- 
lems in mind as he works. The 
designers of the exercises have 
recognized that most participants 
wifil be sufficiency sophisticated 
to guess the kinds of responses 
looked for by the observers. By 
being forced to work under time 
pressure they are thus deprived of 
opportunities to weigh responses 
in favor of what they believe the 
approved behavior to be. 




Can the administrator sell his project? 
The ELA appraisal team in Boston will 
get some or all of the answers on the 
video tapes. 

ELA in the 
Ed. I). Program 

ELA has been designed to elicit 
information about leadership quali- 
ties and the patterns of leadership 
behavior that until now has been 
unavailable to either the educa- 
tional community or to those insti- 
tutions of higher learning engaged 
in the professional training of edu- 
cators. It is anticipated that ELA 
will become a rich source of data 
that can be utilized nationally by 
school systems for effective man- 
agement staffing, staff develop- 
ment, and job assignment, and by 



university graduate schools for 
the development of programs lead- 
ing to management roles in public 
education. 

For participants in the National 
Ed. D. Program, ELA is expected 
to provide information ofi leader- 
ship potential, competencies, 
strengths, and weaknesses that 
will enable both participants and 
the Nova instructional staff to 
identify development needs. In 
the case of the approximately one- 
hundred current participants, who 
took part in the field testing pro- 
gram. Performance Analyses will 
serve for self-assessment only and 
will not at any time enter the par- 
ticipant's Nova file. For future 
clusters, ELA will also be used to 
aid lecturers and others in pro- 
viding the appropriate instruction- 
al components to meet the revealed 
needs of participants. The incor- 
poration of ELA into the National 
Ed. D, Program will thus afford 
participants an additional instru- 
ment for improving competencies 
and is consistent with the Pro- 
gram's purposes — improving 
education by improving the lead- 
ership skills of school managers. 

Decisions regarding the when 
and how of inserting ELA into the 
National Ed. D. Program have 
been made on the basis of two 
considerations. If we view ELA 
as a diagnostic and competency- 




Time pressure is part of the ELA process. 
ERC observers keep exercises on tight 
schedule. 
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Identification mechanism, it be- 
comes evident that the process 
should be applied early if it is to 
have value for the participant. It 
Is only through early appraisal 
that the participant will be enabled 
to draw upon the full range of de- 
velopmental and remedial oppor- 
tunities offered during the three 
years of the Program. 

The second consideration re- 
lates to economics. We have pointed 
out earlier that all developmental 
(continued on p. 8) 




Group exercise finds the participant 
who knows how to influence friends and 
opponents, 

DinKMision / Exercise Matrix 



uimonsion /cXBrcisG Matrix 

demonstrates graphically the 
application of the various 
exercises and simulations to 
the measurement of behavior 
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three dimensions. Within the 
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it is a primary function of the ^ 
exercise to measure the indi- 
cated dimension; a ^ means 
it is a secondary function of 
that exercise. 
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ELA in the Ed.D. Program 
costs of ELA have been borne by 
the Ed. D. Program; implementa- 
tion costs will also be paid for 
out of operating funds, Thus, the 
need to keep application costs 
within manageable limits demands 
that appraisals be conducted on a 
cluster, rather than an individual, 
basis. I 

For both these reasons the 
consensus of the Program's ad- 
ministration is that the earliest 
practical time to institute ELA 
would be concurrently with the 
entry of Group II clusters. While 
there are no plans for applying 
ELA to clusters already in opera- 
tion, ERC will consider scheduling 
appraisals for groups of partici- 
pants willing to form ad hoc clus- 



ters and to meet at a specified 
location for the two days needed 
for the exercises and simulations. 

Since the costs of administer- 
ing an appraisal program are not 
included within the tuition structure 
for currently operating clusters, 
those presently enrolled partici- 
pants who may wish to utilize the 
ELA process must also be pre- 
pared to incur the costs that ERC 
would charge for this additional 
service. 

Apart from its use as an instru- 
ment for assessment of participat- 
ing individuals, what can ELA 
contribute to the National Ed. D. 
Program? Judging from the re- 
sults of our pilot testing with four 
clusters, referred to above, we 
expect that ELA will enable us to 



generate an important body of 
statistical data that will point di- 
rections for modifying or adding 
to the program content, or alter- 
ing the emphasis placed on the 
various components of the pro- 
gram in ways that will extend the 
Program's capabilities for devel- 
oping leadership skills. 

The full potential of ELA — for 
assessment, as an aid in leader- 
ship development, and in per- 
formance evaluation — is yet to be 
explored. We anticipate, however, 
that it will provide an additional 
resource — and one unique to the 
Nova Ed. D. Program — for the 
betterment of education and of 
educators. 
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